Novel Approaches to the Spectral and Colorimetric Color Reproduction

All the different approaches taken for spectral data acquisition can be narrowed down to two main methods; the first one is using spectrophotometer, spectroradiometer, hyper- and multi- spectral camera through which the spectra can be most probably attained with a high level of accuracy in a direct manner. Nonetheless, the price at which the spectra are acquired is very high. However, there is also a second approached in which the spectra are estimated from the colorimetric information. The second approach, even though it is very cost efficient, is of limited level of accuracy, which could be due to the methods or the dissmiliarity of learning and testing samples used. In this work, through looking upon the spectral estimation in a different way, it is attempted to enhance the accuracy of the spectral estimation procedures which is fulfilled by associating the spectral recovery process with spectral sensitivity variability present in both different human observers and RGB cameras. The work is split into two main sections, namely, theory and practice. In the first section, theory, the main idea of the thesis is examined through simulation, using different observers’ color matching functions (CMFs) obtained from Asano’s vision model and also different cameras’ spectral sensitivities obtained from an open database. The second part of the work is concerned with putting the major idea of the thesis into use and is comprised of three subsections itself. In the first subsection, real cameras and cellphones are used. In the second subsection, using weighted regression, the idea presented in this work, is extended to a series of studies in which spectra are estimated from their corresponding CIEXYZ tristimulus values. In the last subsection, obserevers’ colorimetric responses are simulated using color matching. Finally, it is shown that the methods presented in this work have a great potential to even rival multi-spectral cameras, whose equipment could be as expensive as a spectrophotometer

Virtual Cleaning of Works of Art Using Deep Learning Based Approaches

Virtual cleaning of art is a key process that conservators apply to see the likely appearance of the work of art they have aimed to clean, before the process of cleaning. There have been many different approaches to virtually clean artworks but having to physically clean the artwork at a few specific places of specific colors, the need to have pure black and white paint on the painting and their low accuracy are only a few of their shortcomings prompting us to propose deep learning based approaches in this research. First we report the work we have done in this field focusing on the color estimation of the artwork virtual cleaning and then we describe our methods for the spectral reflectance estimation of artwork in virtual cleaning. In the color estimation part, a deep convolutional neural network (CNN) and a deep generative network (DGN) are suggested, which estimate the RGB image of the cleaned artwork from an RGB image of the uncleaned artwork. Applying the networks to the images of the well-known artworks (such as the Mona Lisa and The Virgin and Child with Saint Anne) and Macbeth ColorChecker and comparing the results to the only physics-based model (which is the first model that has approached the issue of virtual cleaning from the physics-point of view, hence our reference to compare our models with) shows that our methods outperform that model and have great potentials of being applied to the real situations in which there might not be much information available on the painting, and all we have is an RGB image of the uncleaned artwork. Nonetheless, the methods proposed in the first part, cannot provide us with the spectral reflectance information of the artwork, therefore, the second part of the dissertation is proposed. This part focuses on the spectral estimation of the artwork virtual cleaning. Two deep learning-based approaches are also proposed here; the first one is deep generative network. This method receives a cube of the hyperspectral image of the uncleaned artwork and tries to output another cube which is the virtually cleaned hyperspectral image of the artwork. The second approach is 1D Convolutional Autoencoder (1DCA), which is based on 1D convolutional neural network and tries to find the spectra of the virtually cleaned artwork using the spectra of the physically cleaned artworks and their corresponding uncleaned spectra. The approaches are applied to hyperspectral images of Macbeth ColorChecker (simulated in the forms of cleaned and uncleaned hyperspectral images) and the \u27Haymakers\u27 (real hyperspectral images of both cleaned and uncleaned states). The results, in terms of Euclidean distance and spectral angle between the virtually cleaned artwork and the physically cleaned one, show that the proposed approaches have outperformed the physics-based model, with DGN outperforming the 1DCA. Methods proposed herein do not rely on finding a specific type of paint and color on the painting first and take advantage of the high accuracy offered by deep learning-based approaches and they are also applicable to other paintings

A Comparison of Colorimetric Performance of Oculus and HTC Virtual Reality Headsets

The colorimetric characterization of the two virtual reality headsets, namely, HTC and Oculus are compared to each other. In order to do that, first, a colorimeter is used to measure the colorimetric values of the primary ramps in a darkened and controlled environment. It is observed that the two headsets behave more or less the same with HTC outputting an overall higher level of luminance and having more consistent right and left displays. Weighted regression is also used as a means to characterize the devices and the results are compared to the traditional method of colorimetric characterization showing the superiority of the weighted regression in this case

Spectral Reflectance Reconstruction Using Fuzzy Logic System Training: Color Science Application

[EN] In this work, we address the problem of spectral reflectance recovery from both CIEXYZ and RGB values by means of a machine learning approach within the fuzzy logic framework, which constitutes the first application of fuzzy logic in these tasks. We train a fuzzy logic inference system using the Macbeth ColorChecker DC and we test its performance with a 130 sample target set made out of Artist's paints. As a result, we obtain a fuzzy logic inference system (FIS) that performs quite accurately. We have studied different parameter settings within the training to achieve a meaningful overfitting-free system. We compare the system performance against previous successful methods and we observe that both spectrally and colorimetrically our approach substantially outperforms these classical methods. In addition, from the FIS trained we extract the fuzzy rules that the system has learned, which provide insightful information about how the RGB/XYZ inputs are related to the outputs. That is to say that, once the system is trained, we extract the codified knowledge used to relate inputs and outputs. Thus, we are able to assign a physical and/or conceptual meaning to its performance that allows not only to understand the procedure applied by the system but also to acquire insight that in turn might lead to further improvements. In particular, we find that both trained systems use four reference spectral curves, with some similarities, that are combined in a non-linear way to predict spectral curves for other inputs. Notice that the possibility of being able to understand the method applied in the trained system is an interesting difference with respect to other 'black box' machine learning approaches such as the currently fashionable convolutional neural networks in which the downside is the impossibility to understand their ways of procedure. Another contribution of this work is to serve as an example of how, through the construction of a FIS, some knowledge relating inputs and outputs in ground truth datasets can be extracted so that an analogous strategy could be followed for other problems in color and spectral science.Samuel Morillas acknowledges the support of the Spanish Ministry of Science under grants PRX17/00384, PRX16/00050 and PID2019-107790RB-C22.Amiri, MM.; Garcia-Nieto, S.; Morillas, S.; Fairchild, MD. (2020). Spectral Reflectance Reconstruction Using Fuzzy Logic System Training: Color Science Application. Sensors. 20(17):1-18. https://doi.org/10.3390/s20174726S118201